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Response to Amendment 

1 . Applicants' response to the last Office Action, filed September 25, 2002 has been 
entered and made of record. 

Claim Objections 

2. Claims 1 ,3,5 are objected to because of the following informalities: in Claim 1 , 
specify that "comprising one or more of:" refers only to the next two limitations, "forming 
one..." and "grouping two..."\ but not to averaging, assigning, setting, and assigning. 
(Otherwise, examiner will simply choose "forming".) Appropriate correction is required. 

Claim Rejections - 35 USC § 103 

3. The following is a quotation of 35 U.S.C. 103(a) which forms the basis for all 
obviousness rejections set forth in this Office action: 

(a) A patent may not be obtained though the invention is not identically disclosed or described as set 
forth in section 102 of this title, if the differences between the subject matter sought to be patented and 
the prior art are such that the subject matter as a whole would have been obvious at the time the 
invention was made to a person having ordinary skill in the art to which said subject matter pertains. 
Patentability shall not be negatived by the manner in which the invention was made. 

4. Claims 1,3,5,21-23 are rejected under 35 U.S.C. 103(a) as being unpatentable 
over Chang (US-PAT-NO: 6741655) in view of Fazzari US-PAT-NO: 5887073, "High 
speed mass flow food sorting apparatus for optically inspecting and sorting bulk 
food products" and further in view of Ricardo de Queiroz, "Mixed Raster Content (MRC) 
Model for Compound Image Compression" (Proc. SPIE 3653, 1 106-1 1 17 (1998)). 

() Regarding Claim 1: 
A method for organizing regions identified in image data, comprising one or more of: 
(examiner chooses forming) 
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forming one or more color clusters (Fazzari Fig5' 96,98,100), each color 
cluster including regions of the image data having a color difference (Fazzari. Fig4) 
that is less than a color threshold; (Fazzari. Fig4: 92,93,94. Col 8 lines 42*45, "...rot 
object 96 in the rot attribute image 88 indicates that the RGB values of the pixels in 
the corresponding locations in color image 91 map into the rot color cloud 92 in RGB 
space as depicted in FIG. 4."). 

grouping two or more regions included in a color cluster into one of one or 
more spatial clusters if closest boundaries of the two or more regions are within a 
distance threshold. 

Fazzari discloses not averaging colors of regions of a spatial cluster to 
generate an average color. 

Chang discloses, averaging colors (mean color) of regions of a spatial cluster 
(region) to generate an average color; 

(Chang. Column 10, line 30, "If a new region is generated from two adjoining regions, its 
mean color is computed 630 by taking weighted average of the mean colors of the two 
old regions.") 

assigning the average color (mean) as a color of the spatial cluster; 
(Chang. Column 10, line 30, "If a new region is generated from two adjoining regions, its 
mean color is computed 630 by taking weighted average of the mean colors of the two 
old regions." 

CoMO line 35, "new region is then assigned one label 650 from the labels of the two old 
regions.") 
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and setting a size of the spatial cluster to a total number of pixels contained in 
the spatial cluster. 

(Chang. Column 10, lines 55-60, "a simplification process 434 is applied to eliminate 
small regions, i.e. regions with less than a given number of pixels." This proves that the 
number of pixels defines sizes of regions.) 

It is desirable to form color clusters as taught by Fazzari for Chang's color 
matching between an input color and a color database, Col 6 line 56 "attribute is 
compared to stored attributes... 301 will be matched 321 against the color database 
31 1", useful in object recognition. Therefore, it would have been obvious to one of 
ordinary skill in the art at the time of the invention to use Fazzari's method of forming 
color clusters in Chang, for object matching. 



Fazzari discloses assigning color clusters 96,98,100 to binary output planes 
88,89,90. Chang and Fazzari disclose everything as described above except, assigning 
each spatial cluster to a binary output plane. 



Queiroz discloses on Page 3 Fig 2, assigning each spatial cluster (R,S in 
to a binary output plane (Fig2 middle picture has 5 binary output planes.) 

It is desirable to apply MRC compression to Chang's individual frame with text 
and graphics for compression and for Color Fax (Queiroz. Abstract). It is also desirable 
to separate text from Chang's frames for Text Match 750. 
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Therefore, it would have been obvious to one of ordinary skill in the art at the time of the 
invention to use Queiroz's MRC compression on individual Chang's frames so they can 
be compressed for Color FAX or to extract text from individual frames for text match. 
() Regarding Claim 3: 

The method of claim 1, further comprising: sorting the spatial clusters according 
to their sizes. 

(Chang. Column 4, lines 31-33, "Extracted regions of video information may be grouped 
based on size." 

Column 10, lines 55-60, "a simplification process 434 is applied to eliminate small 
regions, i.e. regions with less than a given number of pixels.") 

() Regarding Claim 5: 
The method ofclaim-4 claim 1, further comprising: creating a background plane, 
(Fazzari. 89 in Fig5. Queiroz. Yellow plane in middle column of Figure 2.) which 
contains one or more regions of the image data that are not included in any color 
cluster. (Background) 

() Regarding Claim 21: 

A computer-readable medium having computer-readable program code 
embodied therein, the computer-readable program code performing the method of 
claim 1. 

(Queiroz. Page 2, Chapter 1 , "MRC is part of RFC 23019 or TIFF-FX (TIFF for Fax 
extended), the IETF file format proposal for Internet Fax.") 
() Regarding Claim 22: 
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A xerographic marking device using the method of claim 1. 
(Queiroz. Page 2, Chapter 1 , "MRC use for color fax is defined in a forthcoming ITU 
standard 7 ' 8 ".) 

() Regarding Claim 23: 

A digital photocopier using the method of claim 1. 
(Queiroz. Page 2, Chapter 1 , "MRC use for color fax is defined in a forthcoming ITU 
standard 7 ' 8 ". 

Fax is also a photocopier.) 

5. Claims 6,7 are rejected under 35 U.S.C. 103(a) as being unpatentable over of 
Fazzari US-PAT-NO: 5887073, "High speed mass flow food sorting apparatus for 
optically inspecting and sorting bulk food products" in view of Gibson US-PAT-NO: 
6750867, "Image processing apparatus". 

() Regarding Claim 6: 
Gibson discloses, A method for organizing regions identified in image data, comprising: 

dividing an image area of the image data into a plurality of tiles; (Fig2: 22) 
(Abstract: "an image includes a tiling device 60 which divides the image into ... tiles.") 

assigning one or more regions (Fig2: roof, chimney) of the image data to a tile 
(Fig2: top left) based on a location of a bounding box (28,30. Abstract: "polygons". Col5 
line 12, "bounding box") of each of the regions, 
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(Col4 line 10, "light tiles 22A are associated with processor A and all the dark tiles 22B, 
as shown in FIG. 2, are associated with processor B." 

Col4 lines 16-30. "...roof 28, the chimney 30, and the upstairs one of the windows 34 lie 
entirely within a light tile 22A, and so need only be sent to processor or device A.") 

Gibson discloses everything as described above except, "and one or more of: 
forming one or more color clusters by including, in each color cluster, regions of the tile 
that have colors that differ by less than a color threshold; and grouping two or more 
regions included in a first color cluster into a spatial cluster, if closest boundaries of the 
two or more regions are within a distance threshold." 

Fazzari discloses a bounding box for classification, Col10 line 11. 

Fazzari discloses, forming one or more color clusters (Fig5: 96,98,100) by 
including, in each color cluster, regions of the tile that have colors that differ by less than 
a color threshold (Fig4: 92,93,94. Col 8 lines 42-45, "...rot object 96 in the rot attribute 
image 88 indicates that the RGB values of the pixels in the corresponding locations in 
color image 91 map into the rot color cloud 92 in RGB space as depicted in FIG. 4."). 

As Gibson discloses, it is desirable to split an image into tiles, to divide rendering 
workload between several devices as in Fig6 7A & 7B; this speeds up the processing. 
Therefore, it would have been obvious to one of ordinary skill in the art at the time the 
invention was made to use Gibson's method in Fazzari, tile the image and assign 
different objects to different tiles depending on their bounding boxes. 

() Regarding Claim 7: 
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The method of claim 6, further comprising: creating a new color cluster (Fig5: 
96,98, 1 00 or another one) for a first region within the tile if the first region is not included 
in any other color cluster (Fig 4: create another color cloud, if nothing falls into 92-94). 

6. Claims 8,9 rejected under 35 U.S.C. 1 03(a) as being unpatentable over of 
Fazzari US-PAT-NO: 5887073, "High speed mass flow food sorting apparatus for 
optically inspecting and sorting bulk food products" in view of Gibson US-PAT-NO: 
6750867, "Image processing apparatus" as applied to claim 7 above, and further in view of 
Chang (US-PAT-NO : 6741655). 
() Regarding Claim 8: 

Fazzari discloses a second color cluster (Fig4: 92, 93 or 94 or another cloud). 

Fazzari and Gibson disclose everything as described above except, creating a 
new spatial cluster within a second color cluster if a closest boundary of a second 
region of the second color cluster not included in an existing spatial cluster is greater 
than a distance threshold away from a boundary of any other region of the existing 
spatial cluster within the second color cluster. 

Chang discloses, creating a new spatial cluster (region) within a second color 
cluster if a closest boundary of a second region of the second color cluster not included 
in an existing spatial cluster (another spatial cluster) is greater than a distance threshold 
("adjoining regions") away from a boundary of any other region of the existing spatial 
cluster within the second color cluster, (spatial-constrained clustering) 
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(Column 10, line 25, "FIG. 6, an iterative spatial-constrained clustering algorithm 610 is 
utilized, where two adjoining regions with a color distance smaller than a given 
threshold, preferably 225, are merged into one new region 620 until color distances 
between any two adjoining regions are larger than the threshold." If two spatial regions 
belonging to 2 nd color cluster are not adjoining, they will not be merged; this leaves 
some spatial cluster within a 2 nd color cluster intact.) 

As Chang discloses (Col10 lines 23-43), it is desirable to merge adjoining regions 
within the same color threshold. This aids in image segmentation for object tracking 
(Col9 lines 5-10). Therefore it would have been obvious to one of ordinary skill in the 
art at the time of the invention to use Chang's method, spatial-constrained clustering, in 
the method of Fazzari to track objects across several video clips (Chang. Abstract) or 
recognize an object within the same image. 

() Regarding Claim 9: 

The method of claim 8, further comprising: 

a) combining 2x2 blocks of tiles into new tiles; 
(Gibson. 72 in Fig6. Fig2 shows 2x2 tiles.) 

b) one or more of: 

combining color clusters (Fazzari. Fig5: 96,98,100) within a new tile if 
colors of regions of the color clusters within the new tile are less than the color threshold 
(Fig4: 92,93,94) apart; and 
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(Fazzari. Fig4: 92,93,94. Col 8 lines 42-45, "...rot object 96 in the rot attribute image 88 
indicates that the RGB values of the pixels in the corresponding locations in color image 
91 map into the rot color cloud 92 in RGB space as depicted in FIG. 4."). 

combining spatial clusters within the new tile if boundaries of regions of 
the spatial clusters within the new tile are less than the distance threshold apart; 

c) rename the new tiles as tiles; and 

repeat a) - c) until a single tile includes all the regions of the image data. 
(Gibson. 72 in Fig6.) 

() Regarding Claims 10: 

Chang discloses, The method of claim 9, further comprising: 
averaging colors (mean color) of regions of a color cluster or a spatial cluster 
(region) to generate an average color; 

(Chang. Column 10, line 30, "If a new region is generated from two adjoining regions, its 
mean color is computed 630 by taking weighted average of the mean colors of the two 
old regions.") 

assigning the average cobras the color of the corresponding color or spatial 
cluster; and 

(Column 10, line 30, "If a new region is generated from two adjoining regions, its mean 
color is computed 630 by taking weighted average of the mean colors of the two old 
regions." 
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Col10 line 35, "new region is then assigned one label 650 from the labels of the two old 
regions.") 

setting a size of the color or spatial cluster to a total number of pixels contained 
in the spatial cluster. 

(Column 10, lines 55-60, "a simplification process 434 is applied to eliminate small 
regions, i.e. regions with less than a given number of pixels." Col 1 1 line 40 "size, i.e., 
the average number of pixels," This proves that the number of pixels defines sizes of 
regions.) 

7. Claim 13 is rejected under 35 U.S.C. 103(a) as being unpatentable over Fazzari 
US-PAT-NO: 5887073, "High speed mass flow food sorting apparatus for optically 
inspecting and sorting bulk food products" in view of Gibson US-PAT-NO: 6750867, 
"Image processing apparatus". 
() Regarding Claim 13: 

An apparatus for processing regions of image data, comprising: 

a color cluster processor that forms one or more color clusters (Fazzari. Fig5: 

96,98,100) by grouping regions of the image data which are within a color threshold of 

each other, 

(Fazzari. Fig4: 92,93,94; each color cloud is a threshold. Col 8 lines 42-45, "...rot object 
96 in the rot attribute image 88 indicates that the RGB values of the pixels in the 
corresponding locations in color image 91 map into the rot color cloud 92 in RGB space 
as depicted in FIG. 4."). 



Application/Control Number: 10/776,602 Page 12 

Art Unit: 2624 

and/or (examiner chooses or) 

a spatial cluster processor that forms one or more spatial clusters for each of the 
color clusters, regions of the color cluster being included in the spatial cluster when their 
respective bounding boxes are within a distance threshold of each other; and 

a planes generator (Fig5: 64) which creates binary output planes (Fazzari. Fig5. 
88,89,90. Co!8 Iine38, "binary attribute image") based on the color clusters; (Fazzari 
Fig5: 96,98,100) 

Fazzari discloses a bounding box for classification, CoMO line 11. 

Fazzari discloses groups the regions into clusters depending on color of the 
regions, as described above. 

Fazzari discloses everything as described above except, the color cluster 
processor also divides an image area of image data into a set of tiles, and assigns the 
regions of the image data to the tiles based on a location of a bounding box of each of 
the regions. 

Gibson discloses, the color cluster processor also divides an image area of image data 
into a set of tiles (Fig2: 22. (Abstract: "an image includes a tiling device 60 which divides the 
image into . . . tiles."), 

and assigns the regions (Fig2: roof, chimney) of the image data to the tiles based on a 
location of a bounding box (28,30. Abstract: "polygons". Col5 line 12, "bounding box") of each 
of the regions, 

(Col4 line 10, "light tiles 22A are associated with processor A and all the dark tiles 22B, as 
shown in FIG. 2, are associated with processor B." 
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Col4 lines 16-30. "...roof 28, the chimney 30, and the upstairs one of the windows 34 lie entirely 
within a light tile 22A, and so need only be sent to processor or device A.") 

As Gibson discloses, it is desirable to split an image into tiles, to divide rendering 
workload between several devices as in Fig6 7A & 7B; this speeds up the processing. 
Therefore, it would have been obvious to one of ordinary skill in the art at the time the 
invention was made to use Gibson's method in Fazzari, tile the image and assign 
different objects to different tiles depending on their bounding boxes to allow for fast 
parallel processing. 

8. Claim 15,16,17,19,20 rejected under 35 U.S.C. 103(a) as being unpatentable 
over Fazzari US-PAT-NO: 5887073, "High speed mass flow food sorting apparatus for 
optically inspecting and sorting bulk food products" in view of Gibson US-PAT-NO: 
6750867, "Image processing apparatus" as applied to claim 13 above, and further in view of 
Chang (US-PAT-NO: 6741655). 
() Regarding Claim 15: 

The apparatus of claim 13, comprising a tile processor that: 

a) combines 2x2 blocks of tiles into a new tile; 
(Gibson. 72 in Fig6 combines. Fig2 shows 2x2 tiles.) 

b) one or more of: 

1) combines the color clusters from the 2x2 blocks of tiles to form a new 
color cluster in the new tile if colors of regions of the color clusters in the tiles are 
within a color threshold of each other, and 
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2) combines regions of each new color cluster having bounding boxes 
which are less than a distance threshold apart; (combine references) 

c) renames the new tiles as old tiles; and repeats a)-c) until a single tile 
includes all regions in the image data. 
(Gibson. 72 in Fig6 combines tiles.) 

Gibson discloses everything as described above except "2)" limitation. 
Chang discloses, 2) combines regions of each new color cluster (Column 10 line 
30 "new region") having bounding boxes which are less than a distance threshold apart; 

(Chang Column 10, line 25, "FIG. 6, an iterative spatial-constrained clustering algorithm 
610 is utilized, where two adjoining regions with a color distance smaller than a given 
threshold, preferably 225, are merged into one new region 620 until color distances 
between any two adjoining regions are larger than the threshold.") 

As Chang discloses (CoMO lines 23-43), it is desirable to merge adjoining regions 
within the same color threshold. This aids in image segmentation for object tracking 
(Col9 lines 5-10). Therefore it would have been obvious to one of ordinary skill in the 
art at the time of the invention to use Chang's method, spatial-constrained clustering, in 
the combined method of Fazzari and Gibson, to track objects across several video clips 
(Chang. Abstract) or recognize an object within the same image. 

() Regarding Claim 16: 

Fazzari and Gibson do not disclose limitation of claim 16. 
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Chang discloses, a color averager (mean color) that calculates an average color 
value of a either a color cluster based on colors of regions included in the color cluster, 
or a spatial cluster (region) based on colors of regions included in the spatial cluster, 
(Chang. Column 10, line 30, "If a new region is generated from two adjoining regions, its 
mean color is computed 630 by taking weighted average of the mean colors of the two 
old regions.") 

and calculates a size of the color or spatial cluster based on a total number of 
pixels of the regions included in the color or spatial cluster. 
(Column 10, lines 55-60, "a simplification process 434 is applied to eliminate small 
regions, i.e. regions with less than a given number of pixels." Col 1 1 line 40 "size, i.e., 
the average number of pixels," This proves that the number of pixels defines sizes of 
regions.) 

As Chang discloses (Col10 lines 23-43), it is desirable to merge adjoining regions 
within the same color threshold. This aids in image segmentation for object tracking 
(Col9 lines 5-10). Therefore it would have been obvious to one of ordinary skill in the 
art at the time of the invention to use Chang's method, spatial-constrained clustering, in 
the combined method of Fazzari and Gibson, to track objects across several video clips 
(Chang. Abstract) or to recognize objects within the same image. 

() Regarding Claim 17: 
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Fazzari and Gibson disclose all as described above except, The apparatus of 
claim 13, further comprising: an inner blob module which eliminates color and/or spatial 
clusters which are not larger than a predefined threshold size. 

Chang disclose, Column 10, lines 55-60, "a simplification process 434 is applied 
to eliminate small regions, i.e. regions with less than a given number of pixels." 

As Chang says, it is desirable to eliminate small noisy regions Col 1 1 line 40, 
"size... of the determined regions can be utilized to eliminate noisy and unimportant 
regions." This enhances image segmentation for object recognition. Therefore, it 
would have been obvious to one of ordinary skill in the art at the time the invention was 
made to use Chang's method in Fazzari or Gibson, for object detection. 

() Regarding Claim 19: 

The apparatus of claim 13, further comprising: 

a marking module which marks regions (outer perimeter) which do not conform to 
a set of predefined criteria, (outer perimeter is unique for every object) 
(Fazzari. Col6 lines 17-21," A property image for a given attribute image contains single- 
bit values which are enabled (i.e.=1) only at those locations in the original attribute 
image containing pixels that satisfy some "property criterion" such as outer perimeter") 

() Regarding Claim 20: 

Fazzari and Gibson disclose all as described above except, The apparatus of 
claim 13, further comprising: a module which sorts the color and/or spatial clusters 
according to size. 
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Chang. Column 4, lines 31-33, "Extracted regions of video information may be 
grouped based on size." 

As Chang says, it is desirable to group regions based on size, for size database 305, to 
create an object list 335. This enhances image segmentation for object recognition. 
Therefore, it would have been obvious to one of ordinary skill in the art at the time the 
invention was made to use Chang's method in Fazzari or Gibson, for object detection by 
size. 



Allowable Subject Matter 

9. Claims 1 1 ,12,18 are objected to as being dependent upon a rejected base claim, 
but would be allowable if rewritten in independent form including all of the limitations of 
the base claim and any intervening claims. 

10. The following is a statement of reasons for the indication of allowable subject 
matter: Claims 11,18 would be allowable because the prior art does not disclose, 
"selecting a spatial cluster; identifying regions of the selected spatial cluster which are 
completely contained within other regions; and eliminating the identified regions from 
the selected spatial cluster before generating the average color", along with other 
limitations in the claim. 

Claim 12 depends on Claim 11 and would be allowable. 

1 1 . Claim 9 may be allowable if applicant deletes, "one or more of:" because of the 
"combining spatial..." limitation. 
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Conclusion 

12. Prior art made of record and not relied upon is considered pertinent to applicant's 
disclosure. Bottou discloses, "High Quality Document Image Compression with DjVu" 
(AT&T Labs, Lincroft, NJ, July 13, 1998). Brecher discloses, "Automated defect 
classification system" (US-PAT-NO: 5544256). Leshem, "Apparatus and method for 
measurement and temporal comparison of skin surface images" (US-PAT-NO: 
6215893). Chang (US-PAT-NO: 6741655). 

1 3. Any inquiry concerning this communication or earlier communications from the 
examiner should be directed to Max Shikhman whose telephone number is (571) 270- 
1669; FAX 571-270-2669. The examiner can normally be reached on Monday-Friday 
8:30AM-6:00PM EST. 

If attempts to reach the examiner by telephone are unsuccessful, the examiner's 
supervisor, JINGGE WU can be reached on (571) 272-7429. The fax phone number for 
the organization where this application or proceeding is assigned is 571-273-8300. 
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Information regarding the status of an application may be obtained from the 
Patent Application Information Retrieval (PAIR) system. Status information for 
published applications may be obtained from either Private PAIR or Public PAIR. 
Status information for unpublished applications is available through Private PAIR only. 
For more information about the PAIR system, see http://pair-direct.uspto.gov. Should 
you have questions on access to the Private PAIR system, contact the Electronic 
Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a 
USPTO Customer Service Representative or access to the automated information 
system, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000. 



Max Shikhman 
8/8/2007 



